A Novel Low-Bit Quantization Strategy for Compressing Deep Neural Networks
نویسندگان
چکیده
منابع مشابه
Learning Accurate Low-Bit Deep Neural Networks with Stochastic Quantization
Low-bit deep neural networks (DNNs) become critical for embedded applications due to their low storage requirement and computing efficiency. However, they suffer much from the non-negligible accuracy drop. This paper proposes the stochastic quantization (SQ) algorithm for learning accurate low-bit DNNs. The motivation is due to the following observation. Existing training algorithms approximate...
متن کاملCompressing Deep Convolutional Networks using Vector Quantization
Deep convolutional neural networks (CNN) has become the most promising method for object recognition, repeatedly demonstrating record breaking results for image classification and object detection in recent years. However, a very deep CNN generally involves many layers with millions of parameters, making the storage of the network model to be extremely large. This prohibits the usage of deep CN...
متن کاملDeepThin: A Self-Compressing Library for Deep Neural Networks
As the industry deploys increasingly large and complex neural networks to mobile devices, more pressure is put on the memory and compute resources of those devices. Deep compression, or compression of deep neural network weight matrices, is a technique to stretch resources for such scenarios. Existing compression methods cannot effectively compress models smaller than 1-2% of their original siz...
متن کاملHfh: Homologically Functional Hashing for Compressing Deep Neural Networks
As the complexity of deep neural networks (DNNs) trends to grow to absorb the increasing sizes of data, memory and energy consumption has been receiving more and more attentions for industrial applications, especially on mobile devices. This paper presents a novel structure based on homologically functional hashing to compress DNNs, shortly named as HFH. For each weight entry in a deep net, HFH...
متن کاملCompressing deep neural networks using a rank-constrained topology
We present a general approach to reduce the size of feedforward deep neural networks (DNNs). We propose a rankconstrained topology, which factors the weights in the input layer of the DNN in terms of a low-rank representation: unlike previous work, our technique is applied at the level of the filters learned at individual hidden layer nodes, and exploits the natural two-dimensional time-frequen...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
ژورنال
عنوان ژورنال: Computational Intelligence and Neuroscience
سال: 2020
ISSN: 1687-5265,1687-5273
DOI: 10.1155/2020/7839064